Various types of color filter of an image pickup element such as a CCD sensor or a CMOS sensor have been used, and examples of such a color filter include a color filter having a combination of primary colors (red, green, and blue) and a color filter having a combination of complementary colors (cyan, magenta, and yellow).
FIG. 13 is a diagram illustrating a primary-color Bayer arrangement of an image pickup element. Red (R) and blue (B) are diagonally arranged in a pixel matrix of 2×2, green (G1 and G2) is diagonally arranged in the remaining two pixels, and this pattern is repeated.
When an object includes a high frequency component which exceeds resolution capability of the image pickup element, aliasing is generated in an image signal generated by the image pickup element due to an adverse effect of the high frequency component. Therefore, various methods for suppressing aliasing have been proposed. For example, a method using a combination of two luminance signals generated in different ways has been proposed in order to suppress generation of aliasing.
One of the luminance signals is generated only using signals corresponding to G (G1, G2) pixels without using signals corresponding to R and B pixels. First, values of signals other than signals corresponding to the G pixels among the signals corresponding to the R, G, and B pixels obtained by digitalizing a signal output from the image pickup element having the primary-color Bayer arrangement are set to 0. Next, a vertical lowpass filter (V-LPF) process which restricts a band in a vertical direction and a horizontal lowpass filter (H-LPF) which restricts a band in a horizontal direction are performed. By this, signals of pixels which have been subjected to compensation using the signals corresponding to the G pixels are generated and a luminance signal of G is obtained. Hereinafter, a luminance signal obtained by performing compensation on pixels which do not correspond to a certain color using signals corresponding to the certain color is referred to as a first luminance signal.
Alternatively, values of signals other than signals of the R pixels are set to 0 and similarly the V-LPF process and the H-LPF process are performed to thereby generate a luminance signal of R. Similarly, values of signals other than signals of the B pixels are set to 0 and similarly the V-LPF process and the H-LPF process are performed to thereby generate a luminance signal of B. Then, the luminance signals of R and B are added to the luminance signal of G, and a resultant signal may be referred to as a first luminance signal.
The other luminance signal is generated using signals of all the colors of the primary-color Bayer arrangement shown in FIG. 13. The V-LPF process which restricts a band in the vertical direction and the H-LPF which restricts a band in the horizontal direction are performed on the signals corresponding to the pixels of all the R, G, and B colors which are obtained by digitalizing the signals output from the image pickup element having the primary-color Bayer arrangement without distinguishing the colors so that a signal is newly obtained. Hereinafter, such a luminance signal obtained using the signals of all the colors without distinguishing the colors is referred to as a second luminance signal.
FIG. 14 is a diagram illustrating spatial frequency characteristics in which the first and second luminance signals can be resolved. An x axis denotes a frequency space in a horizontal (H) direction of an object and a y axis denotes a frequency space in a vertical (V) direction of the object. The further a point is located from an intersection between the x axis and the y axis, the higher a spatial frequency in the point is.
Resolution limits in the horizontal and vertical directions of the first luminance signal generated only using the signals corresponding to the G pixels are equal to a Nyquist frequency (π/2) of an arrangement of the G pixels. However, since some diagonal lines do not include the G pixels, a limit resolution frequency in a diagonal direction is lower than those in the horizontal and vertical directions and an inside portion of a region 1401 having a diamond shape shown in FIG. 14 corresponds to a spatial frequency in which the first luminance signal can be resolved. Since, among the R, G, and B luminance signals, the G luminance signal obtained only using the signals corresponding to the G pixels has the highest resolution, even when the first luminance signal is generated by synthesizing the R, G, and B luminance signals with one another, the same spatial frequency in which the first luminance signal can be resolved is obtained.
On the other hand, since the second luminance signal is generated using the signals corresponding to all the color pixels, when the object is achromatic, an outer square region 1402 shown in FIG. 14 corresponds to a spatial frequency in which the second luminance signal can be resolved. Unlike the first luminance signal, since any one of the color pixels is included in all lines diagonally extending, a spatial frequency in a diagonal direction in which the second luminance signal can be resolved is higher than that of the first luminance signal. However, when a red object is captured, for example, signals output from the pixels other than the R pixels are negligible. Accordingly, only a resolution corresponding to a region 1403 which is a quarter of the region corresponding to the achromatic object is obtained.
Taking characteristics of the first and second luminance signals described above into consideration, a configuration for suppressing aliasing included in an image signal by generating a luminance signal has been proposed. For example, a configuration for generating a luminance signal by changing a mixing ratio of the first and second luminance signals depending on a determination as to whether an object is achromatic or chromatic has been proposed (refer to Patent Literature 1). Furthermore, a configuration for generating a luminance signal by changing a mixing ratio of the first and second luminance signals depending on a degree of the diagonal correlation of an object shown in FIG. 14 has been proposed (refer to Patent Literature 2).
However, although these methods are useful in terms of the suppression of aliasing, noise signals other than the aliasing are not suppressed. For example, in recent years, miniaturization of pixels of image pickup elements has been developed. Therefore, noise may be increased due to the miniaturization of pixels. Although various methods for suppressing such noise by performing signal processing have been proposed, image blur is generated when such noise is suppressed, which is an adverse effect.
To address this problem, a method for suppressing noise by dividing an image signal into a plurality of frequency components has been proposed (refer to Patent Literature 3). Furthermore, a method for suppressing noise by generating an image signal by reducing an image signal and synthesizing the reduced image signal with the original image signal has been proposed (Patent Literature 4).
Specifically, a reduction process is performed on a signal of an input image so that a reduced image including a frequency component lower than that of the input image is generated. Then, edge strength is detected using the reduced image signal having the low frequency component, and a region in which an edge component is to be maintained is obtained in accordance with the edge strength. Weights of regions are changed so that an image included in the region in which the edge component is to be maintained is not blurred and the original image signal and the reduced image signal having the low frequency component are synthesized with each other to thereby newly generate an image signal.